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Method for segmenting and estimating a moving object motion 



(54 

(57) A method for use in an object-oriented analy- 
sis-synthesis coder, for segmenting an image of a cur- 
rent frame into moving objects and describing each 
motion of the moving objects with a set of motion 
parameters, which comprises the steps of: (a) compar- 
ing the current frame with a preceding frame to detect a 
motion vector field; (b) sliding window blocks on the 
motion vector field, and computing the degree of homo- 
geneity for each of the window blocks to produce a seed 
block; (c) determining a set of initial values for the set of 
motion parameters based on the seed block; (d) evalu- 
ating whether each motion vector contained in and 
around the seed block is describable with the set of ini- 



tial values or not, to thereby detect a revised region; (e) 
determining a set of revised values for the set of motion 
parameters from the revised region; (f) evaluating 
whether each motion vector contained in and around 
the revised region is describable with the set of revised 
values or not, to thereby detect a newly revised region; 
and (g) repeating the steps (e) and (f) until the differ- 
ence between a current revised region and a previous 
revised region is less than a predetermined value, to 
thereby determine a set of final values for the set of 
motion parameters. 
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Description 

Field of tlie Invention 

5 The present invention relates to an objected-oriented analysis-syntliesis coding technique for encoding image sig- 

nals; and, more particularly, to a method for segmenting an image according to moving objects therein and estimating 
each motion of the moving objects. 

Description of the Prior Art 

10 

In digital television systems such as video-telephone, teleconference and high definition television systems, a large 
amount of digital data is needed to define each video frame signal since a video line signal in the video frame signal 
comprises a sequence of digital data referred to as pixel values. Since, however, the available frequency bandwidth of 
a conventional transmission channel is limited, in order to transmit the large amount of digital data therethrough, it is 

15 inevitable to compress or reduce the volume of data through the use of a data compression technique, especially in the 
case of such low bit-rate video signal encoder as video-telephone or teleconference system. 

One of such methods for encoding video signals for a low bit-rate encoding system is the so-called object-oriented 
analysis-synthesis coding technique, wherein an input video image containing moving objects therein is divided accord- 
ing to the moving objects; and each object is described by three sets of parameters defining the motion, contour(or 

20 shape) and pixel data of the object. 

In order to achieve the primary goal of the object-oriented analysis-synthesis coding technique, i.e., the task of 
exactly dividing the input video image into the moving objects and precisely estimating each motion of the moving 
objects, there have been proposed several approaches. One of them is a hierarchically structured segmentation tech- 
nique fsee Michael Hotter et a!., "Image Segmentation Based on Object Oriented Mapping Parameter Estimation", Sig- 

25 nal Processing . 15. No. 3, pp 315-334(October 1988)). 

According to the hierarchically structured segmentation technique, an area of the input video image to be seg- 
mented is defined by a set of uniform motion and position parameters denoted as mapping parameters. For instance, 
eight parameters are used to describe an arbitrary three-dimensional motion of a planar rigid object. In a first step, a 
change detector distinguishes between temporally changed and unchanged regions of two successive fields. Each 

30 changed image region is interpreted as one object. The motion and position of each object is described by one set of 
mapping parameters. Based on the mapping parameters and the information on a temporally preceding field, recon- 
struction of a temporally ensuing field can be achieved. In a next step of hierarchy, those regions of the image, which 
are not correctly described in their mapping, are again detected by the change detector and treated according to the 
changed parts detected at the first step of hierarchy. However, in this technique, if a large number of moving objects is 

35 included within one changed area, it may be difficult to properly segment them. 

Summary of the Invention 

It is, therefore, a primary object of the invention to provide an improved method for exactly segmenting an image 
40 according to the moving objects therein and precisely estimating each motion of the nfx)ving objects. 

In accordance with the invention, there is provided a method, for use in an object-oriented analysis-synthesis coder, 
for segmenting an image of a current frame into moving objects and describing each motion of the moving objects with 
a set of motion parameters, which comprises the steps of: 

45 (a) comparing the current frame with a preceding frame to detect a moving area in the current frame; 

(b) further comparing the current frame with the preceding frame to detect a motion vector for each pixel in the mov- 
ing area, to thereby produce a motion vector field; 

(c) sliding window blocks of an N x N pixel size on the motion vector field, N being a positive integer, and computing 
the degree of homogeneity for each of the window blocks to select a most homogeneous window block as seed 

50 block; 

(d) determining a set of initial values for the set of motion parameters based on the seed block; 

(e) evaluating whether each motion vector contained in and around the seed block is describable with the set of ini- 
tial values or not, to thereby detect a revised region defined by motion vectors described with the set of initial val- 
ues; 

55 (f) determining a set of revised values for the set of motion parameters from the revised region; 

(g) evaluating whether each motion vector contained in and around the revised region is describable with the set of 
revised values or not, to thereby detect a newly revised region defined by motion vectors described with the set of 
revised values; 
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(h) repeating the steps (f) and (g) until the difference between a current revised region and a previous revised 
region is less than a predetermined value, to thereby determine a set of final values for the set of motion parame- 
ters; and 

(1) repeating the steps (c) to (h) until each set of motion parameters for each of the moving objects is determined. 

5 

Brief Description of the Drawings 

The above and other objects and features of the present invention will become apparent from the following descrip- 
tion of preferred embodiments given in conjunction with the accompanying drawings, in which: 

10 

Fig. 1 is a block diagram of segmenting an image into moving objects and describing each motion of the moving 
objects with a set of motion parameters in accordance with the present invention; and 

Fig. 2 shows a flow chart illustrating an operational flow of the seed block detection and region growing circuit 
shown in Fig. 1. 

15 

Detailed Description of the Preferred Embodiments 

Fig. 1 describes a block diagram of segmenting moving objects and describing each motion of the moving objects 
with a set of motion parameters in accordance with the present invention. 

20 As shown in Fig. 1 , two input signals, i.e., a current frame signal and a reconstructed previous frame signal are fed 
to a change detector 100 and a motion estimator 102, respectively 

The change detector 100 compares the current frame signal with the reconstructed previous frame signal to detect 
a changed area from the background in the current frame. The detected changed area information is sent to the motion 
estimator 102 and a motion vector field detector 103. 

25 The motion estimator 102 detects motion vectors for a set of selected pixels, i.e., feature points, in the changed 
area, wherein each of the feature points is a pixel capable of representing its neighboring pixels. That is, a number of 
feature points are first selected from all of the pixels contained in the changed area. Then, motion vectors for each of 
the selected feature points are determined by forming a block of a predetermined size at each of the selected feature 
points and by using a block matching algorithm(see, e.g., J. R. Jain et al., "Displacement Measurement and Its Appli- 

30 cation in Interframe Image Coding", IEEE Transactions on Communications . COM -29 . No. 12 pp 1 799-1 808(December 
1981)), wherein each of the motion vectors represents a spatial displacement between one feature point in the current 
frame and a corresponding matching point, i.e.. a most similar pixel, in the reconstructed previous frame. 

The motion vector field detector 103 determines motion vectors for non-feature points in the changed area by aver- 
aging the motion vectors for the feature points, to thereby detect motion vectors for all of the pixels in the changed area. 

35 The changed area defined by motion vectors for all of the pixels therein is called a motion vector field. The information 
on the motion vector field is sent to a seed block detection and region growing circuit 104. 

The seed block detection and region growing circuit 104 segments the motion vector field into moving objects and 
describes each motion of the moving objects with a set of motion parameters. In accordance with the present invention, 
in order to describe each motion of the moving objects, there is employed an 8-parameter motion model (see. e.g.. Gilad 

40 Adiv, "Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects", 
IEEE Transactions on Patent Analysis and Machine Intelligence . PAMI-7, No. 4, pp 384-401 (July 1985)), which maybe 
represented as follows: 

Vx(x,y) = 3^ + a2X + a3y + a7X^ + agxy Eq. (1) 

45 

Vy(x,y) = a4 + agx + a gy a7xy + agy^ Eq. (2) 

wherein Vx(x,y) is the horizontal component of a motion vector at a position (x,y) and Vy(x,y) is the vertical component 
of the motion vector at the position (x,y). 

50 Referring to Fig. 2, the eight motion parameters Ibr each of the moving objects in the motion vector field are deter- 
mined through two processes: a seed block detection process 10 and a region growing process 20. 

The seed block detection process 10 detects a seed block in order to determine a set of initial values for the eight 
motion parameters, and the region growing process 20 increases the size of the seed block through the use of an iter- 
ative region growing algorithm, to thereby estimate a set of actual values for the eight motion parameters. 

55 In the seed block detection process 1 0, there are two steps 1 0 and 12. In the step 1 0, a W x W pixel size of window 
block slides on the motion vector field; and the degree of homogeneity for each position of the window block is com- 
puted. Thereafter, in the step 12, a most homogeneous window block is selected as a seed block. The degree of homo- 
geneity may be determined as follows: 
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10 



m+w-^ n+w-^ 

^(ai.* • •.^8)= £ e(x,y;ai,* • -.ag) Eq. (4) 

X=m y=n 

e{x.y:a,. • • • .as)={V^{x.y)-V x(x.y))\{Vy{x.y)-V y{x.y)f Eq. (5) 

2 

^x(^'y) = 5i + + 83/ + ayX + agJ^K Eq. (6) 

Vy{x,y) = 34 + + as/ + a-^xy + a^y^ Eq. (7) 

15 wherein D(m,n) represents the degree of homogeneity at a window position (m,n), W is a window size and {a^ *, 

Bq*} is an optimal set of motion parameters for a window block. The optimal set of motion parameters is a solution for a 

set of eight equations, each of which is derived by taking partial derivatives of Eq. (4) with respect to ai, a2 as and 

equating them to 0. As shown in Eqs. (3) to (7), the degree of homogeneity for a window block is determined through 
the use of an error function between a given set of motion vectors in a window block and an estimated set of motion 

20 vectors obtained from the optimal set of motion parameters. 

After detecting the seed block, the process 20 begins to determine a set of initial values for a set of motion param- 
eters. And then, in order to correct the initial values to an actual set of motion parameters, a convergence process is 
taken through the use of a region growing technique described below. 

In step 14, a set of initial values for a set of motion parameters is determined based on the seed block detected in 

25 step 1 2 or a set of revised values for the set of motion parameters is determined based on a revised region detected in 
step 16 below. 

The set of initial values for the set of motion parameters are determined by fetching the optimal set of motion 
parameters for the seed block. And, the set of revised values for the set of motion parameters is determined based on 
all of the motion vectors contained in the revised region as in the case of detecting the optimal set of motion vectors for 
30 the seed block. 

In step 16, in order to correct the initial values or the revised values to an actual set of motion parameters, a region 
growing process is carried out by evaluating whether each motion vector contained in and around the seed block or the 
revised region is capable of being described or not with the initial values or the revised values, to thereby detect the 
revised region or define a newly revised region. 
35 In step 18, the difference between a current revised region and a previous revised region is compared with a pre- 
determined value, to decide whether a region growing iteration is to be continued. The region growing iteration is 
repeated until the difference between the current revised region and the previous revised region is less than the prede- 
termined value, to thereby determine a set of final values for the set of motion parameters. 

After detecting the final values for the set of motion parameters, another set of motion parameters is detected by 
40 repeating the steps 10 to 1 8 until each set of motion parameters for each of the moving objects is determined. 

While the present invention has been shown and described with respect to the particular embodiments, it will be 
apparent to those skilled in the art that many changes and modifications may be made without departing from the spirit 
and scope of the invention as defined in the appended claims. 

45 Claims 

1, A method, for use in an object-oriented analysis-synthesis coder, for segmenting an image, said image containing 
moving objects, of a current frame into the moving objects therein and describing each motion of the moving 
objects with a set of motion parameters, which comprises the steps of: 

50 

(a) comparing the current frame with a preceding frame to detect a moving area in the current frame; 

(b) further comparing the current frame with the preceding frame to detect a motion vector for each pixel in the 
moving area, to thereby produce a motion vector field; 

(c) sliding window blocks of an N x N pixel size on the motion vector field, N being a positive integer, and com- 
55 puting the degree of homogeneity for each of the window blocks to select a most homogeneous window block 

as a seed block; 

(d) determining a set of initial values for the set of motion parameters based on the seed block; 
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(e) evaluating whether each motion vector contained in and around the seed block is describable with the set 
of initial values or not, to thereby detect a revised region defined by motion vectors described with the set of 
initial values; 

(f) determining a set of revised values for the set of motion parameters from the revised region; 

(g) evaluating whether each motion vector contained in and around the revised region is describable with the 
set of revised values or not, to thereby detect a newly revised region defined by motion vectors described with 
the set of revised values; 

(h) repeating the steps (f) and (g) until the difference between a current revised region and a previous revised 
region is less than a predetermined value, to thereby determine a set of final values for the set of motion 
parameters; and 

(i) repeating the steps (c) to (h) until each set of motion parameters for each of the moving objects is deter- 
mined. 
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